Predictive Inference

Suppose we are given a random vector Y consisting of n observations with joint density fn(· ; θ) depending on a p-dimensional parameter θ. We write `n(θ) for the log likelihood, `n(θ) = log fn(Y ; θ) where Y are the actual observations. The most familiar case is when Y consists of independent observations Y1, ..., Yn with common density f0(· ; θ); henceforth, this is called the i.i.d. case. In that case, `n(θ) = ∑n r=1 log f0(Yr ; θ) is the sum of n i.i.d. components. However, our general formulation does not require that Y consist of independent or identically distribution observations, provided `n(θ) and its derivatives obey certain laws of large numbers and central limit theorems which are analogous to those that arise in the i.i.d. case. There are some general conventions that we make at the outset and, generally, stick to throughout this work. Many of the notations and conventions are the same as those introduced by McCullagh (1987). We use superscripts to denote components of θ, such as θi for the ith component. Where we use scalar functions of θ, such as ψ(θ) or Q(θ) in the following discussion, subscripts will indicate differentiation with respect to the components of θ. Thus Qi = ∂Q ∂θi , ψij = ∂2ψ ∂θi∂θj , etc. For derivatives of the log likelihood itself, we use the letter U with appropriate subscripts, e.g.